Computer Estimation of Spoken Language Skills

Skillful performance in a human language often involves a composite of elementary skills, such that language skills, cognitive skills, and social skills can be conflated in the judgement of a human listener. A new, computer-based method (PhonePass testing) provides an estimate of spoken language skills, relatively independent of the speaker's other social and cognitive skills. The PhonePass system has been implemented using interactive voice response to administer a standardized spoken English test that measures speaking and listening during a 10-minute telephone call. The system calculates scores on five performance subscales from a set of more basic measures that are produced by automatic speech recognition of examinee responses. Item response theory is used to analyze and scale aspects of examinee performance. The scores are also related to performance rubrics used in criterion-based human scoring of similar responses. The test construct is outlined, and the scaling methods and validation process are described with reference to experimental procedures that were used in the development of the SET-10 test, a standardized instrument.